Eye-State Action Unit Detection by Gabor Wavelets

Eyes play important roles in emotion and paralinguistic communications. Detection of eye state is necessaryfor applications such as driver awareness systems. In this paper, we develop an automatic system to detect eye-state action units (AU) based on Facial Action Coding System (FACS) by use of Gabor wavelets in a nearly frontal-viewed image sequence. Three eye-state AU (AU 41, AU42, and AU43) are detected. After tracking the eye corners in the whole sequence, the eye appearance information is extracted at three points of each eye (i.e., inner corner, outer corner, and the point between the inner corner and the outer corner) as a set of multi-scale and multi-orientation Gabor coefficients. Then, the normalized Gabor coefficients are fed into a neural-network-based eyestate AU detector. An average recognition rate of 83% is obtained for 112 images from 17 image sequences of 12 subjects.